Step 2: Create and run a Log Stream Staging task

Once the Log Stream target endpoint is created, a Log Stream Staging task must be defined that will replicate any changes to the original source endpoint data and metadata to the Log Stream Staging folder.

The Log Stream Staging task creates a “Staging” file in the Log Stream Staging folder. This file contains the changes from the source database transaction log as a compressed binary file.

Before running the Log Stream Staging task, you must define and add the source endpoint where your data is currently stored and the target endpoint where you want to replicate the data – in this case, the Log Stream target endpoint. After you add the endpoints to your task, you can select the tables from the source endpoint that you want to replicate. For information on selecting tables, see Selecting tables and/or views for replication.

To define a Log Stream Staging task:

1. In the Tasks view, click New Task. The New Task dialog box opens.
2. Specify a name for the task.

3. Choose the Log Stream replication profile option.

   

4. Click OK.

1. The next step in the Log Stream task is to select the source endpoint where your data is currently stored, and the Log Stream target endpoint.

The following figure provides an example of selected source and target endpoints for a Log Stream Staging task:

1. In the right pane in the Console, click Table Selection to select the source tables that you want to replicate.

   The task will create the metadata in the Log Stream staging folder and then capture the changes from the specified source endpoint.

   The task is now ready to run.

2. To execute the Log Stream Staging Task, click the Run button.

   Information note

   The Log Stream Staging task must be run before the Replication task described in Step 3: Create a replication task to transfer the staged data to the target.

Log Stream Staging limitations and considerations

• The Retrieve all source columns on UPDATE option in Task Settings > Change Processing > Change Processing Tuning is only available for Log Stream Staging tasks.
• The Full Load and Store Changes replication options are not relevant.
• In the Advanced Run Options dialog box, only the following options are supported:
  • Tables are already loaded. Start processing changes from: Date and Time.
  • Recovery: Recover using locally stored checkpoint
• In the Task Settings window, under the Target Metadata tab:

  • Target table schema is unavailable

  • The Allow unlimited LOB size option is disabled.
  • Limit LOB size to (KB): Default is 8 KB; maximum permitted size is 102,400 KB

• In the Task Settings window, the following tabs and/or settings are not available:
  • Metadata > Control Tables tab
  • Full Load tab
  • Under the Change Processing tab: Apply Changes Settings tab, and Store Changes Settings tab.
  • In the Change Processing Tuning tab: Batch Tuning and Miscellaneous Tuning settings
  • Change processing mode: only Transactional apply is available
  • Error handling: Data Errors tab, Table Errors tab, Apply Conflicts tab
  • Character Substitution tab
• In the Table Settings window:
  • The following tabs are not available: Transform, Filter, Parallel Load, and LOB Column Handling
  • In the General tab, the Map to target table section is unavailable.
• Global Transformations are not available.
• The source_lookup Data Enrichment function is not supported. For more information on this function, see Data Enrichment functions
• Source endpoints that do not support CDC (such as, ODBC, Hadoop) cannot be used as sources for Log Stream Staging tasks.
• File Channel cannot be used as a source for a Log Stream Staging task.
• When the source endpoint is IBM DB2 for z/OS or IBM DB2 for iSeries, the option in the Tables settings' Transform tab for changing the character set or data type for a specific input column is not supported.